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Abstract 

A thousand graduates of nineteen Canadian universities, in a variety of disciplines, re¬ 
sponded to an invitation to name the professors they remembered as excellent teachers 
and to say why. Analyses of their comments reveal what they considered to be character¬ 
istics of effective university teaching. Comparisons of various sub-groups suggest that 
there are some differences in the patterns of good teaching between anglophone and 
francophone faculties and between fields of study-humanities, social sciences, biological 
sciences and physical sciences—but the patterns are basically similar in most university 
settings. 

As a contribution to the improvement of teaching in the universities of Canada, a recent 
study sought to identify professors** who were effective teachers, and to discover from 
them and from their former students what was characteristic of them and their teaching. 

Deans and alumni directors provided the names and addresses of roughly seven thous¬ 
and graduates of twenty-four faculties and schools in nineteen universities—members of 
the classes of 1968 (the previous year), 1963 (five years earlier) and 1958 (ten years 
earlier). Among them they represented the fourteen top faculties in terms of numbers of 
first degrees awarded in 1966—67, the latest year for which data were published at the 
time. There were four of arts, three of science, two each of education, engineering, com¬ 
merce/administration, law and medicine, and one each of nursing, physical education, 
agriculture, home economics, pharmacy, dentistry and library science. Some were in large 
universities, some in small. Five of the twenty-four were French-language faculties, nine¬ 
teen were English-language, and the list included at least one university in every one of the 

* Edward F. Sheffield is Professor of Higher Education, University of Toronto 

** As is customary in Canada and the United States, the term “professor” is used here to include univer¬ 
sity teachers of all ranks. 
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ten provinces, from Memorial University of Newfoundland to the University of British 
Columbia. 

Letters were sent to these seven thousand graduates inviting them to name professors 
they remembered as excellent teachers, and to say what there was about those professors 
and their teaching that made them effective. It was explained that of the teachers so 
identified a score or more would be invited to write essays on the teaching of under¬ 
graduate students, reflecting their own beliefs and practices, and that these would be 
published. (Copies of the letter and attachments form Appendix A.) Replies were received 
from just over a thousand. On the average each named two professors; some named as 
many as six. The number of different professors identified as excellent teachers was 
1,002. Forty-one graduates took the trouble to send back the response form saying that, 
unfortunately, they had had no professor whom they would classify as an excellent teacher. 

After applying a simple weighting scale to take account of the differences in class 
size and the order in which professors’ names appeared on the returned forms, in each 
faculty there was identified at least one, in most cases several, of those mentioned with 
sufficient frequency to suggest that they were excellent teachers. Because only one was 
sought in each faculty, the final choice was to some extent arbitrary. Twenty-four profes¬ 
sors so chosen were invited to contribute essays on their teaching. Twenty-three did so 
and their essays have been compiled as a book entitled Teaching in the Universities: No 
One Way (Sheffield, 1974). 

The chief purpose of this paper is to present the results of the analyses of comments 
on effective teaching made by the thousand responding graduates. 

Earlier it was reported that seven thousand graduates were invited to participate in the 
study. To be precise, they numbered 6,928, and 1,014 (14.6%) responded to the invitation. 
The percentage of response was highest (15.9%) for the class of ’68, lower (14.2%) for 
the class of ’63, and still lower (11.4%) for the class of ’58. The range of participation 
rates varied greatly from one faculty to another—from less than ten per cent in five cases 
to more than twenty per cent in seven. (A note on the representativeness of the sample is 
to be found in Appendix B.) Some indication of the interest of those who responded is 
seen in the fact that 88% of them signed their names and gave their addresses in order that 
they might receive a report on the results of the project. 

When the request to graduates was being planned, it was decided to invite free com¬ 
ment on the characteristics, qualities, methods and procedures which, in their view, 
identified each of the professors they named as an excellent teacher. They could have 
been asked, instead, to check a list of possible characteristics phrased in advance by the 
investigator. The replies received indicated that the decision taken yielded a wealth of 
factors and forms of expression which would have been obscured by a checklist. But 
there was posed the problem of classifying the responses. These were grouped, on the 
basis of personal judgment and therefore somewhat imprecisely, into four main categories 
and 53 sub-categories (see Table 1). 

The data provided by the responding graduates were subjected to a number of analy¬ 
ses and comparisons. Those which are reported here are: 
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analysis of the overall pattern for all 24 faculties and schools; 
comparison of faculties of arts and science with professional schools; 
comparison of anglophone and francophone faculties; 
comparison by size of university; 

comparison of humanities, social sciences, biological sciences and physical sciences; 

comparison of pure and applied studies; 

comparison by course level; 

comparison by class size; 

comparison by year of graduation; 

comparison of undergraduates and graduates in arts and science. 

The essentials of these analyses follow. Table 1 shows the overall pattern of the char¬ 
acteristics identified. It will be described in detail later. Perhaps of most immediate 
interest is the list of characteristics most frequently mentioned. Here are the top ten, in 
order of frequency of mention: 

1 Master of his subject, competent 

2 Lectures well prepared, orderly 

3 Subject related to life, practical 

4 Students’ questions and opinions encouraged 

5 Enthusiastic about his subject 

6 Approachable, friendly, available 

7 Concerned for students’ progress, etc. 

8 Had a sense of humour, amusing 

9 Warm, kind, sympathetic 

10 Teaching aids used effectively 

The Overall Pattern 

As has been mentioned, consideration was given at the beginning to the possibility of ma¬ 
king a list of perhaps a score or more characteristics of effective university teaching which 
might be checked by the graduates to indicate which applied to the professors they re¬ 
membered as excellent teachers. This was discarded, however, in favour of an open-ended 
kind of question which would leave the graduates free to say what came to their minds 
about the teachers they remembered. What resulted was a fantastic range of words des¬ 
criptive of teachers and their ways of teaching. The responses were grouped first into four 
main categories: 


A 

Personal qualities or attributes 

N 

1,758 

% 

19 

B 

Subject mastery, scholarship, devotion to teaching 

1,603 

18 

C 

Attitudes toward and relations with students 

1,295 

14 

D 

Methods and procedures 

4,390 

48 


course content and organization 
presentation of material 
other 


Each of these was then divided into a number of sub-categories, as discrete as possible. 
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Inevitably there was a good deal of overlapping, not only between sub-categories within 
main categories but also between categories. It could have been avoided, or at any rate it 
could seem to have been avoided, if a check-list had been provided in the beginning, but 
then who would know what would have been said by the graduates if they had not been so 
guided? 

Table 1 lists the four categories and 53 sub-categories, with additional items provided 
for “other comment” (by graduates who named excellent teachers but did not say why) 
and “no comment” (for the graduates who responded only to say that they had met no 
good teacher while they were undergraduates). The number of comments by graduates in 
each sub-category was tallied and expressed also as a percentage of the total number of 
comments recorded. It is of significance to note that if a particular point about a professor 
was made more than once by a graduate, that point was tallied only once. To illustrate— 
where a graduate described a former teacher as being kind and, later, as having a warm 
personality, one tally only was recorded for sub-category 5. 

Table 2 presents these characteristics of effective university teaching in order of the 
frequency with which they were mentioned by responding graduates. Attention has been 
drawn already to the first ten, each of which accounts for at least 3.0% of the total num¬ 
ber of comments. Including these, 28 items (SC 13, consisting of “other comments,” is 
omitted from this count) were mentioned at least 100 times by respondents. It is perhaps 
needful to point out again, however, that the comments presented are descriptive rather 
than prescriptive. If graduates had been asked what they thought the characteristics of 
effective university teaching should be the pattern might have been somewhat different. It 
certainly would have omitted the critical comments which were in fact made. 

Another way of presenting the data is to tally the number and calculate the percentage 
of professors for whom each characteristic was mentioned by at least one responding gra¬ 
duate. The results of this analysis do not differ significantly from those shown in Charts 1 
and 2 so they are not spelled out here. 

Characteristics of Effective Teaching Identified in Other Studies 

In a study based on the opinions of undergraduates at the State University of New York 
at Albany in 1966 (Musella and Rusch, 1968), students were asked “what teacher beha¬ 
viors promoted their thinking.” Classification of 395 responses to this open-ended ques¬ 
tion indicated that the top ten were: 

1 Attitudes toward subject (enthusiastic, interested) 

2 Attitudes toward students (sympathetic, helpful, actively concerned) 

3 Questioning (effective use of questions) 

4 Speaking ability (skilled in presenting material) 

5 Knowledge of subject (exceedingly well informed) 

6 Organization of subject matter (systematic and thoroughly organized) 

7 Discussion (extensive and effective) 

8 Ability to explain (explanations clear and to the point) 

9 Tolerance to [sicj disagreement (encourages and values reasonable disagreement) 

10 Instructor as “human being” (attractive personality) 
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Table 1 

Characteristics of Effective University Teaching 

Graduates’ Comments — All 24 Faculties and Schools (arranged in order of code numbers) 


Code Categories Sub-categories Comments 

No. % 


1 

Personal 

Dynamic, vivacious 

260 

2.8 

2 

qualities or 

Dramatic 

4 

0.0 

3 

attributes 

Intelligent, brilliant 

110 

1.2 

4 

A 

Mature, wise 

8 

0.1 

5 


Warm, kind, sympathetic 

278 

3.0 

6 


Had a sense of humour, amusing 

321 

3.5 

7 


Humble, modest 

60 

0.7 

8 


Flexible, open-minded 

53 

0.6 

9 


Honest, sincere, genuine 

101 

u 

10 


Fair, impartial 

43 

0.5 

11 


Conscientious, thorough 

166 

1.8 

12 


Organized, logical, efficient 

90 

1.0 

13 


Other (including critical) comments 

264 

2.9 

14 

Subject 

Master of his subject, competent 

717 

7.8 

15 

mastery. 

F.vcr learning, up to date, inquiring 

132 

1.4 

16 

scholarship. 

Experienced in his field 

133 

1.5 

17 

devotion to 

Enthusiastic about his subject 

385 

4.2 

18 

teaching 

Broad knowledge and interests 

59 

0.6 

19 

B 

Loved teaching, gave it high priority 

163 

1.8 

20 


Other comments 

14 

0.2 

21 

Attitudes 

Respect for students as persons 

142 

1.6 

22 

toward and 

Concerned for students’ progress, etc. 

325 

3.6 

23 

relations 

Demanding of hard work, excellence 

155 

1.7 

24 

with 

Sensitive to students’ feelings 

214 

2.3 

25 

students 

Approachable, friendly, available 

372 

4.1 

26 

C 

Sought social contacts with students 

27 

0.3 

27 


Interested in student activities 

15 

0.2 

28 


Other (including critical) comments 

45 

0.5 

29 

Methods and 

Students expected to do much on their own 

145 

1.6 

30 

procedures 

Attitudes, principles stressed over facts 

69 

0.8 

31 

D 

Subject related to life, practical 

555 

6.1 

32 

-course 

Approach to subject critical, scientific- 

66 

0.7 

33 

content and 

professor expressed own point of view 

55 

0.6 

34 

organization 

Course planned at students’ level 

138 

1.5 

35 


Course goals and expectations clear 

42 

0.5 

36 


Course organized, systematic 

251 

2.7 

37 


Summaries and reviews provided 

28 

0.3 

38 


Other comment on course organization 

38 

0.4 

39 


Lectures well prepared, orderly 

712 

7.8 

40 


Voice audible, well-modulated, etc. 

93 

1.0 

41 


Use of language, vocabulary, notable 

83 

0.9 

42 

-presenta¬ 

Lectures oratorical, dramatic 

55 

0.6 

43 

tion of 

Professor rarely used notes; informal 

175 

1.9 

44 

material 

Main points stressed; outline provided 

187 

2.0 

45 


Professor questioned, used Socratic method 

92 

1.0 

46 


Students’ questions, opinions encouraged 

481 

5.3 

47 


Teaching aids used effectively 

287 

3.0 

48 


Professor experimented with methods 

45 

0.5 

49 


Other (including critical) comments 

87 

1.0 

50 


Small-group arrangements used well 

257 

2.8 

51 


Assignments reasonable, helpful 

195 

2.1 

52 


Grading fair, innovative 

253 

2.8 

53 


Other comments on methods and procedures 

10 

0.1 

54 

Other 





comment 


8 

0.1 

55 

No comment 


89 

1.0 


9,143 


100.1 
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Table 2 

Characteristics of Effective University Teaching 

Graduates’ Comments - All 24 Faculties and schools (arranged in order of frequency) 


Code 

Category * 

Sub-category 

Comments 




No. 

% 

14 

B 

Master of his subject, competent 

717 

7.8 

39 

D 

Lectures well prepared, orderly 

712 

7.8 

31 

D 

Subject related to life, practical 

555 

6.1 

46 

D 

Students’ questions, opinions encouraged 

481 

5.3 

17 

B 

Enthusiastic about his subject 

385 

4.2 

25 

C 

Approachable, friendly, available 

372 

4.1 

22 

C 

Concerned for students’ progress, etc. 

325 

3.6 

6 

A 

Had a sense of humour, amusing 

321 

3.5 

5 

A 

Warm, kind, sympathetic 

278 

3.0 

47 

D 

Teaching aids used effectively 

278 

3.0 

13 

A 

Other comments on personal qualities or 





attributes 

264 

2.9 

1 

A 

Dynamic, vivacious 

260 

2.8 

50 

D 

Small-group arrangements used well 

257 

2.8 

52 

D 

Grading fair, innovative 

253 

2.8 

36 

D 

Course organized, systematic 

251 

2.7 

24 

C 

Sensitive to students’ feelings 

214 

2.3 

51 

D 

Assignments reasonable, helpful 

195 

2.1 

44 

D 

Main points stressed; outline provided 

187 

2.0 

43 

D 

Professor rarely used notes; informal 

175 

1.9 

11 

A 

Conscientious, thorough 

166 

1.8 

19 

B 

Loved teaching, gave it high priority 

163 

1.8 

23 

C 

Demanding of hard work, excellence 

155 

1.7 

29 

D 

Students expected to do much on their own 

145 

1.6 

21 

C 

Respect for students as persons 

142 

1.6 

34 

D 

Course planned at students’ level 

138 

1.5 

16 

B 

Experienced in his field 

133 

1.5 

15 

B 

Ever learning, up to date, inquiring 

132 

1.4 

3 

A 

Intelligent, brilliant 

110 

1.2 

9 

A 

Honest, sincere, genuine 

101 

1.1 

40 

D 

Voice audible, well-modulated, etc. 

93 

1.0 

45 

D 

Professor questioned, used Socratic method 

92 

1.0 

12 

A 

Organized, logical, efficient 

90 

1.0 

55 


No comment 

89 

1.0 

49 

D 

Other comments on presentation of material 

87 

1.0 

41 

D 

Use of language, vocabulary, notable 

83 

0.9 

30 

D 

Attitudes, principles stressed over facts 

69 

0.8 

32 

D 

Approach to subject critical, scientific 

66 

0.7 

7 

A 

Humble, modest 

60 

0.7 

18 

B 

Broad knowledge and interests 

59 

0.6 

33 

D 

Professor expressed own point of view 

55 

0.6 

42 

D 

Lectures oratorical, dramatic 

55 

0.6 

8 

A 

Flexible, open-minded 

53 

0.6 

28 

C 

Other comments on relations with students 

45 

0.5 

48 

D 

Professor experimented with methods 

45 

0.5 

10 

A 

Fair, impartial 

43 

0.5 

35 

D 

Course goals, expectations clear 

42 

0.5 

38 

D 

Other comments on course organization 

38 

0.4 

37 

D 

Summaries and reveiws provided 

28 

0.3 

26 

C 

Sought social contacts with students 

27 

0.3 

27 

C 

Interested in student activities 

15 

0.2 

20 

B 

Other comments on subject mastery, etc. 

14 

0.2 

53 

D 

Other comments on methods and procedures 

10 

0.1 

4 

A 

Mature, wise 

8 

0.1 

54 

- 

Other comment 

8 

0.1 

2 

A 

Dramatic 

4 

0.0 


Categories: A (1-13) 
B (14-20) 
C (21-28) 
D (29-53) 
Source: Table 1 


Personal qualities or attributes 
Subject mastery, scholarship, devotion to teaching 
Attitudes toward and relations with students 
Methods and procedures 
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“Although the students were asked to indicate characteristics promoting thinking,” 
explain Musella and Rusch, “it is not inconceivable that the responses of the students 
were really responses to what characterizes ‘most effective’ teaching.” 

With specific reference to qualities important for teaching in general, the students were 
asked to rank ten pre-formulated items. The five thought to be most important by the 
students are listed below in order of importance: 

1 Expert knowledge of subject 

2 Systematic organization of subject matter 

3 Ability to explain clearly 

4 Enthusiastic attitude toward subject 

5 Ability to encourage thought 

A study of Criteria of Effective Teaching in an Institution of Higher Education by the 
Office of Institutional Research of the University of Toledo (1967), produced sixty criteria 
of effective teaching arranged in rank order on the basis of ratings by students, faculty 
and alumni of that University. The first dozen were as follows: 

1 Being well prepared for class 

2 Establishing sincere interest in the subject being taught 

3 Demonstrating comprehensive knowledge of his subject 

4 Using teaching methods which enable students to achieve objectives of the course 

5 Constructing tests which search for understanding on the part of the students rather 
than rote memory ability 

6 Being fair and reasonable to students in evaluation procedures 

7 Communicating effectively at levels appropriate to the preparedness of students 

8 Encouraging intelligent independent thought by students 

9 Organizing the course in logical fashion 

10 Motivating students to do their best 

11 Treating student with respect 

12 Acknowledging all questions to the best of his ability 

Two years later the same Office (Toledo, 1969) tested its findings in three other uni¬ 
versities: New Mexico State University, Northern Illinois University and Western Kentucky 
University. The first eleven criteria of the original study were the first eleven in the second, 
although in a slightly different order. Based in part on the Toledo studies, the University 
of British Columbia (1970) developed a list of criteria for use in making its Master Teacher 
Award: 

1 Having a comprehensive knowledge of the subject 

2 Being habitually well prepared for class 

3 Having enthusiasm for the subject 

4 Having the capacity to arouse interest in it among the students 

5 Establishing good rapport with the students both in and out of classes 

6 Encouraging student participation in class 

7 Setting a high standard and successfully motivating students to try to attain such a 
standard 
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8 Communicating effectively at levels appropriate to the preparedness of students 

9 Utilizing methods of evaluation of student performance which search for understand¬ 
ing of the subject rather than just ability to memorize. 

Hildebrand, Wilson and Dienst (1971) identified characteristics which distinguished 
best from worst teachers and summarized the characteristics which were peculiar to the 
best: 

1 Has command of the subject, presents material in an analytic way, contrasts various 
points of view, discusses current developments, and relates topics to other areas of know¬ 
ledge. 

2 Makes himself clear, states objectives, summarizes major points, presents material in 
an organized manner, and provides emphasis. 

3 Is sensitive to the response of the class, encourages student participation, and welcomes 
questions and discussion. 

4 Is available to, and friendly towards students, is interested in students as individuals, is 
himself respected as a person, and is valued for advice not directly related to the course. 

5 Enjoys teaching, is enthusiastic about his subject, makes the course exciting, and has 
self-confidence. 

As a result of a survey of students at the University of Bradford, Smithers (1969) re¬ 
ported that “students expect a lecturer to be able to lecture. In particular, they look to 
him for expertise and lucidity. They also want lecturers to thoroughly prepare their lec¬ 
tures, to give them an obvious structure and to respond to questions.” 

In his review of research on the student-teacher relationship in the college classroom, 
Wagner Thielens, Jr. (1971) reports: 

The student portrait of the ideal professor is relatively consistent from campus to campus. 
Thorough subject knowledge is the trait most widely sought .... Other highly regarded 
qualities are the teacher’s capacity to convey information and ideas clearly, his ability to 
organize his presentation well, and his enthusiasm for his topic. 

All of which suggests that the characteristics of effective teaching revealed in this 
study are similar to those usually identified. 

Comparisons 

In the comparisons which follow, emphasis is placed on those items or characteristics 
which, like the ten most frequently mentioned in the overall pattern (page 10 above), 
are at or above the three per cent level.* This kind of selection is intended to make it easi¬ 
er to comprehend the data by simplifying the presentation. Complete data are available 
on request from the author. 

Comparison of Faculties of Arts and Science with Professional Schools 

In this comparison, thirteen items are at or above the three per cent level either for 


* An item at or above the three per cent level is one for which at least three per cent of the total number 
of comments were recorded, i.e. at least 274 out of the total of 9,143. 
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faculties of arts and science (N = 3,362) or for professional faculties and schools (N = 
5,781). In no case is one of these items mentioned twice as often by one group as by the 
other; in no case even fifty per cent more often. 

It would appear, therefore, that between these two groups there is no substantial dif¬ 
ference in the patterns of identified characteristics of effective teachers and teaching. 


Comparison of Francophone and Anglophone Faculties 

There are five francophone faculties in the total sample (N = 1,143): sciences de l’educa- 
tion and sciences de Fadministration at l’Universite Laval, droit and chirurgie dentaire at 
l’Universite de Montreal, and arts at FUniversite d’Ottawa. These are compared with the 
five most similar anglophone faculties (N = 2,024) in the project: Education at the Uni¬ 
versity of Alberta, Commerce at Sir George Williams University, Law at the University of 
British Columbia, Medicine at Dalhousie University, and Arts at St. Dunstan’s University. 
In this case fourteen items are at or above the three per cent level. In two instances there 
are more than twice as many comments by one group as by the other, and in another 
seven instances (six, when SC 13 consisting of “other comments” is excluded) one set of 
comments is at least fifty per cent more numerous than the other. Thus for eight of the 
fourteen items selected for scrutiny the difference is worthy of examination. 

Those mentioned markedly more often by graduates of the anglophone faculties were: 


Sub-category 

6 Had a sense of humour, amusing 
25 Approachable, friendly, available 
50 Small-group arrangements used well 
17 Enthusiastic about his subject 
22 Concerned for students’ progress, etc. 
47 Teaching aids used effectively 


Ratio 

(3.1 to 1) 
(almost 2.0 to 1) 
(1.8 to 1) 

(1.7 to 1) 

(1.6 to 1) 

(1.6 to 1) 


Those mentioned much more often by graduates of the francophone faculties were: 

16 Experienced in his field (2.8 to 1) 

31 Subject related to life, practical (1.7 to 1) 

There are real differences between these two groups. Can they be explained? Would it 
be appropriate to suggest that in the case of the francophone graduates there seems to 
have been more experience of distance between the lecturing professor and his students 
than in the case of the anglophone graduates? If so, was this really a difference in experi¬ 
ence or only a difference in those parts of the experience which were thought worth men¬ 
tioning? Are the differences culture-linked? 


Comparison by Size of University 

The nineteen universities in the study were divided into three groups by the number of 
full-time students enrolled in 1967-68: Large—more than 10,000 (Alberta, British Colum¬ 
bia, McGill, Montreal and Toronto), medium-sized—5,000 to 10,000 (Carleton, Laval, 
Manitoba, McMaster, Ottawa, Queen’s, Saskatchewan, Sir George Williams and Western 
Ontario) and small—less than 5,000 (Dalhousie, Memorial, New Brunswick, Nova Scotia 



16 


STO A 


Technical and St. Dunstan’s). (The numbers of comments recorded by group are 4,167, 
3,915 and 1,061.) When responses from graduates of these three groups are compared, 
there are sixteen sub-categories at or above the three per cent level. If the highest in each 
group of three is related to the lowest, there is one item (SC 44) for which the ratio is 
greater than 2 to 1, and one other for which it is at least 1.7 to 1 :* 


Sub-category 

Ratio 

Large 

Medium Small 

44 Main points stressed, outline provided 

(2.4 to 1) 

1.0 

2.4 

6 Had a sense of humour, amusing 

(1.7 to 1) 

1.7 

1.0 


Sub-category 44 (main points stressed; outline provided) is most characteristic of the 
small universities, least of the large. The reverse is true of SC 6 (had a sense of humour, 
amusing). 

It must be concluded that few differences in characteristics of effective teaching seem 
to be related directly to university size. 


Comparison of Humanities, Social Sciences, Biological Sciences and Physical Sciences 
When graduates named professors whom they remembered as good teachers, they indica¬ 
ted what courses they had taken from them, e.g. History 15, Pharmacology 443. The data 
by subject were grouped by field-humanities (N = 2,033), social sciences (N = 3,796), 
biological sciences (N = 1,533), physical sciences (N = 1,781)—in accordance with the 
classification used by the Dominion Bureau of Statistics (now Statistics Canada) in its re¬ 
ports on Salaries and Qualifications of Teachers in Universities and Colleges (see, for ex¬ 
ample, the report for 1968-69, Tables 8-11). For eighteen sub-categories there were com¬ 
ments at or above the three per cent level. When one relates the highest and lowest per¬ 
centage of comments in each group of four, it is seen that there are three for which the 
ratio is at least 2 to 1, and another six (making half of those at the three per cent level) 
for which it is at least 1.7 to 1. 


Sub 

-category 

Ratio 

Human¬ 

Social 

Biological 

Physical 




ities 

sciences 

sciences 

sciences 

44 

Main points stressed, 
outline provided 

(6.5 to 1) 

1.0 



6.5 

15 

Ever learning, up to 
date, inquiring 

(3.8 to 1) 

1.0 


3.8 


34 

Course planned at 
students’ level 

(2.8 to 1) 

1.0 

1.0 


3.4 


The next six items in order of difference in frequency of mention among the four 
fields are: 


*When comparing pairs of series, ratios of at least 1.5 to 1 are drawn to the attention of the reader as 
indicating differences worthy of mention. Such a ratio is more likely when highs and lows of more 
than two series are being compared. For this reason, in such cases only ratios of at least 1.7 to 1 are 
pointed out. 
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6 

Had a sense of humour, 

(almost 





amusing 

2.0 to 1) 

2.0 


1.0 

17 

Enthusiastic about 
his subject 

(1.9 to 1) 

1.9 

1.0 

1.0 

47 

Teaching aids used 
effectively 

(1.8 to 1) 

1.0 


1.8 

36 

Course organized, 
systematic 

(1.8 to 1) 

1.0 

1.8 


39 

Lectures well prepared, 
orderly 

(1.7 to 1) 

1.0 

1.7 


46 

Students’ questions, 
opinions encouraged 

(1.7 to 1) 

1.7 

1.0 



It was in the physical sciences that item 44 (main points stressed; outline provided) 
was mentioned with relatively great frequency, and in the humanities in which it was 
seldom mentioned. Indeed, on this point there is a gradation down from physical to bio¬ 
logical to social sciences to humanities. On item 34 (course planned at students’ level) the 
physical sciences are high again, with the other three fields about equal, and much lower. 
With respect to sub-category 15 (ever learning, up to date, inquiring) mention is noticeably 
more frequent in the biological sciences than in the other three fields. 

There seem to be substantial differences in the patterns of characteristics of effective 
teaching among these fields. It is not obvious, however, that the patterns can be described 
and differentiated in general terms. 

Comparison of Pure and Applied Studies 

The DBS classification of fields of study used for the data on which the previous compari¬ 
son is based was used also to classify responses into two other dimensions: pure and appli¬ 
ed studies (N = 5,226, 3,917). The comments made about a chosen professor who taught 
French, for example, were classified as “pure” while those about a professor who taught 
soil mechanics were classified as “applied.” This comparison for the items at or above the 
three per cent level shows that in no case is the percentage of comments as much as twice 
as great for one series as the other. In only one case, with respect to sub-category 6 (had a 
sense of humour, amusing), was the ratio as high as 1.5 to 1, and indeed it was no higher 
than that—the advantage being to the pure studies series. 

The conclusion to which this comparison leads is that the patterns observed in pure and 
applied studies are for all practical purposes the same. 

Comparison by Course Level 

Tire information provided by responding graduates made it possible to know whether a 
mentioned professor had been met at the level of one, two, three, four, five, six, seven or 
eight years beyond the level of junior matriculation. For example, a course in the first 
year of Arts at St. Dunstan’s (where the level of entry was that of junior matriculation) 
would be classified as level 1, while a course in the third year of Education at Alberta 
(where the level of entry was that of senior matriculation) would be at level 4. The num¬ 
bers at levels 1, 6, 7 and 8 were relatively small, so level 1 is grouped with level 2, and 
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levels 6, 7 and 8 are combined. (Total numbers of comments for the five resulting cate¬ 
gories, from 1-2 up, are 1,838,2,000,2,162, 1,697 and 1,397.) 

Here we find no item for which the ratio of responses is greater than 1.7 for the high¬ 
est to 1 for the lowest. At 1.7 to 1 there are two: 


Sub-category 

17 Enthusiastic 
about his 
subject 

6 Had a sense of 
humour, amu¬ 
sing 


Ratio Years 
1-2 

(1.7 to 1) 

(1.7 to 1) 


Year Year 

3 4 

1.7 1.0 

1.7 


Year Years 

5 6-7-8 


1.0 


For sub-category 17, levels 1-2 and 3 are high, and levels 4 to 8 low. Much the same is 
true of sub-category 6, although in this case level 6-7-8 too is low. 

These differences are minor, suggesting that patterns of effective teaching do not vary 
in accordance with course level. 


Comparison by Class Size 

The data were originally grouped in accordance with seven ranges of class size: Up to 10, 
11-30, 31-60, 61-90, 91-120, 121-200, and over 200. In order to create groupings large 
enough to allow valid comparison, the first two classes were joined, as were the last two. 
(Numbers of comments in the four ranges of class size, from small to large, are 3,563, 
2,998, 1,282 and 1,256.) 

There are three items for which differences in the frequency of comments are as great 
as 2 to 1 and they are worthy of note: 


Sub-category 

Ratio 

Up to 30 

31-60 

61-120 Over 120 

36 Course organized, 

systematic 

(2.7 to 1) 

1.0 


2.7 

6 Had a sense of humour, 

amusing 

(2.0 to 1) 


1.0 

2.0 

52 Grading fair, innovative 

(2.0 to 1) 

2.0 


1.0 


Items 36 and 6 were both mentioned with noticeably more frequency by graduates 
commenting on professors met in the largest classes (over 120). Were these characteristics 
actually more common in such large classes, or is it just that they were especially appre¬ 
ciated when found there? On the other hand, item 52 was more common as a comment 
on professors met in the smallest classes, with a fairly regular gradation from small classes 
to large. Is fairness in grading thought by students to be a function of the ratio of students 
to professors, within single classes? 

There are no other sub-categories for which the differences are as great as 1.7 to 1. 

It would seem, then, that although the differences in patterns of characteristics of ef¬ 
fective teaching by class size are not general, there are a few items on which the largest 
classes are noticeably different from the smallest. 
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Comparison by Year of Graduation 

The possibility that graduates’ views of their teachers might change over the years follow¬ 
ing departure from the university, or that there might be changing styles in the apprecia¬ 
tion of good teaching, accounts for the choice of three graduating classes at intervals of 
five years as participants in the study. As has been mentioned elsewhere, the latest class 
for which names and addresses were available when the project was begun was that of 
1968 (N = 5,330). To it, then, were added the classes of 1963 (N = 2,452) and 1958 
(N = 1,350). This meant that some members of the class of ’58 were commenting on pro¬ 
fessors with whom they had studied as many as fifteen years earlier. 

The outstanding trait of the class of ’58 is that they remembered their good teachers 
as persons who were demanding of hard work, excellence (SC 23); that is the one item for 
which the percentage of comments is more than twice as great by members of one gradu¬ 
ating class than another, in this case a ratio of 2.6 for the class of’58 to 1 for the class of 
’68. There are only two other items for which ratios are as great as 1.7 to 1.: 


Sub-category 

Ratio 

1958 

1963 

1968 

23 Demanding of hard work, excellence 

(2.6 to 1) 

2.6 


1.0 

5 Warm, kind, sympathetic 

(1.7 to 1) 

1.7 


1.0 

52 Grading fair, innovative 

(1.7 to 1) 

1.0 


1.7 


There is a higher proportion of comments about the warmth, kindness and sympathy 
of their teachers (SC 5) by members of the class of’58 than by members of the two later 
classes, whereas comments on fair or innovative grading (SC 52) are most common in the 
case of the class of’68, less common for the class of’63 and least common for the class of 
’58. 

On the whole, therefore, it appears that there are some but not many differences asso¬ 
ciated with year of graduation, or years since graduation. Were the teachers of members 
of the class of ’58 really warmer, kinder and more demanding, or had a legend time to 
grow? It is easier to observe the relation between fair or innovative grading and years 
since graduation because there is no doubt that more emphasis has been placed recently 
on this feature of university teaching. 

Comparing Undergraduates and Graduates in Arts and Science 

In the fall of 1968, when plans for this study were being shaped, a draft of the question¬ 
naire intended for graduates was distributed in an appropriately modified form to a class 
of 139 students in the third year of arts and science at the University of Toronto. They 
were told the purpose of the questionnaire and invited to complete it and comment on it 
in order to test and improve it. The essential difference between the form as they received 
it and the form which went later to graduates across the country was that the under¬ 
graduates were invited to name the one professor whose teaching they thought was best 
among those under whom they had studied in their first two years at the University. (It 
was at their suggestion that the final form of the questionnaire invited graduates to name 
one or more professors whose teaching was remembered as being especially effective.) 

The comments of the arts and science undergraduates (N = 808) were compared with 
those of the graduates of the seven faculties of arts and science (N = 3,362) included in the 
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project. There are only four items for which the difference in percentage of comments is 
greater than 1.4 to 1, but in these cases the difference is marked, ranging from 2.9:1 as 


follows: 

Sub-category 

Ratio 

Graduates 

Undergraduates 

40 Voice audible, 
well-modulated 

(2.9 to 1) 

1.0 

2.9 

47 Teaching aids used 
effectively 

(2.4 to 1) 

1.0 

2.4 

14 Master of his subject, 
competent 

(2.4 to 1) 

2.4 

1.0 

44 Main points stressed, 
outline provided 

(1.9 to 1) 

1.0 

1.9 


It was the graduates who stressed subject mastery (SC 14) much more than the under¬ 
graduates, but in the other three cases highlighted here the undergraduates were those 
with the higher percentages of comment. One can speculate on the possible reasons for 
these differences. The data provided by the undergraduates revealed, for one thing, that 
few of them had had experience of classes of fewer than 100 students in their first two 
years, so almost all of the teachers they mentioned had been met in large classes. Another 
factor which may be meaningful is that the majority of the students, and the professors 
they mentioned with greatest frequency, were in science courses, for which audio-visual 
aids were not only appropriate but available. It is possible, therefore, that the comparison 
attempted here is not wholly valid, that it is more accurately described as one between 
classes of first and second year in predominantly science courses and classes throughout 
the range of course levels from 1 to 5 in courses divided between arts and science. 


Summary and Confirmation 

Of the nine comparisons made above there are two which show marked and probably 
significant differences in pattern—between francophone and anglophone faculties and 
among the four main fields of study: humanities, social sciences, biological sciences and 
physical sciences. In another two cases there are some differences worthy of mention— 
among class sizes and according to graduating year. Another comparison—between under¬ 
graduates and graduates in arts and science—may belong in this group but there is some 
doubt whether the data available are sufficiently relevant to make the comparison valid. 
Finally, there are four comparisons which yield little or no evidence of pattern differ¬ 
ences—between faculties of arts and science and professional schools, among universities 
of different size, between pure and applied studies, and by course level. 

These observations are confirmed by the calculation of rank order correlations between 
several of the series of comments on characteristics of effective teaching. Rank correlation 
was calculated for the two comparisons mentioned in the previous paragraph as showing 
“marked” differences in pattern, one of the two showing “some” differences, and three 
of the four said to “yield little or no evidence of pattern differences.” The results are 
shown below. 
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Pairs of series compared 


Rank correlation 

Comparisons showing "marked”differences in pattern 


Francophone faculties 

Anglophone faculties 

.59 

Humanities 

Social sciences 

.78 

Humanities 

Biological sciences 

.67 

Humanities 

Physical sciences 

.73 

Social sciences 

Biological sciences 

.63 

Social sciences 

Physical sciences 

.71 

Biological sciences 

Physical science 

.72 

Comparisons showing “some 

” differences in pattern 


Class of ’58 

Class of ’63 

.82 

Class of’58 

Class of ’68 

.76 

Class of ’63 

Class of ’68 

.81 

Comparisons showing ‘‘little’ 

’ difference in pattern 


Faculties of arts and science 

Professional faculties and schools 

.79 

Large universities 

Medium-sized universities 

.83 

Large universities 

Small universities 

.80 

Medium-sized universities 

Small universities 

.82 

Pure studies 

Applied studies 

.79 


Note that, with the exception of the comparisons between francophone and anglo¬ 
phone faculties and all but one of those between fields of study, rank correlations are in 
the neighborhood of .80, indicating much agreement or, to put it in the terms used in this 
analysis, little difference in patterns of comments. Even the comparison with the least cor¬ 
relation (.59) indicates considerable agreement. 

Of the ten characteristics drawing three per cent or more of graduates’ comments in the 
overall pattern (see Table 2), nine appeared at or above the three per cent level in all of 
the sub-divisions made for comparisons, and the tenth appeared in all but one. On none of 
these was there agreement in all comparisons. Those which came nearest to commending 
complete agreement, no matter what sub-divisions of comments were made, were these six: 


Sub-category 

14 Master of his subject, competent 
5 Warm, kind, sympathetic 
25 Approachable, friendly, available 
31 Subject related to life, practical 
39 Lectures well prepared, orderly 
46 Students’ questions, opinions encouraged 


Agreement in 

8 out of 9 cases 
7 out of 8 cases 
7 out of 9 cases 
7 out of 9 cases 
7 out of 9 cases 
7 out of 9 cases 


The one item among the top ten in the overall pattern on which there was marked 
disagreement when data were sub-divided for the making of comparisons was sub-category 


* Corrected for ties. Standard error at the .60 level is .15: at the .70 level, .14; at the .80 level, .11. 
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6 (had a sense of humour, amusing). With respect to this characteristic there was a dif¬ 
ference of at least 1.5 to 1 in six cases out of nine. 

In sum, it can be said of the comparisons that, although there are a few exceptions, the 
characteristics of effective undergraduate teaching are basically similar in most university 
settings. 

Conclusion 

This study has not revealed new information about the general characteristics of effective 
university teaching. Rather, it has underlined what was already known about teaching in 
some countries, notably the U.S.A., Britain and Australia. It is thought, however, to be 
the first such nation-wide study in Canada and, therefore, the first time it has been possi¬ 
ble to document the fact that the overall pattern of good teaching in Canadian universities 
is like that of good teaching elsewhere. 

The comparisons of patterns of comments made by various sub-groups of graduates 
are more novel than the data on the overall pattern. They too stress the universality of the 
basic characteristics of effective undergraduate teaching. It must be remembered, how¬ 
ever, that the patterns reported here are descriptive of good university teachers collective¬ 
ly rather than individually. In developing his own style, any one professor may depart 
noticeably from the overall pattern and still be an effective teacher. 
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Appendix A 


Letter Sent to Graduates, 
Inviting their Assistance in the Project 



UNIVERSITY OF TORONTO 
TORONTO. ONTARIO. CANADA 


Innis College, May 1969. 


Dear University Graduate: 

This request for assistance in a research project is being addressed 
to a representative sample of men and women who graduated from Cana¬ 
dian universities in 1958, 1963 and 1968. 

As the first step in an investigation of the characteristics of 
effective teaching in university, 1 am asking recent graduates to 
help identify professors (of whatever rank, including lecturers 
and instructors) who, in their view, are excellent teachers. Of the 
teachers so identified, a score or more will be invited to write essays 
on the teaching of undergraduate students, reflecting their own beliefs 
and practices. These essays and a digest of the views of the graduates 
who have responded to this request will be published. 

It would be greatly appreciated if you would take the 10 or 15 minutes 
required to complete the attached simple questionnaire, and then send 
it to me in the enclosed envelope. 

If you care to add your name and address, I shall be glad to keep 
you informed of the results. 


Yours sincerely, 



Edward F. Sheffield, 

Professor of Higher Education. 


EFS:jac. 
Enc. 
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List of Universities and Faculties whose Graduates are being Invited to Assist in the Study 


University 

University of Alberta 
University of British Columbia 
Carleton University 
Dalhousie University 
Universite Laval 


University of Manitoba 
McGill University 
McMaster University 
Memorial University of Newfoundland 
Universite de Montreal 
University of New Brunswick 
Nova Scotia Technical College 
Universite d’Ottawa 

Queen’s University at Kingston 
St. Dunstan’s University 
University of Saskatchewan 
Sir George Williams University 
University of Toronto 
University of Western Ontario 


Faculty or School 
Education 

Home Economics; Law 

Science 

Medicine 

Sciences de l’administration 
(Commerce); 

Sciences de Teducation (Peda- 
gogie, Orientation) 
Agriculture 
Arts 

Physical Education 
Science 

Chirurgie dentaire; Droit 

Science 

Engineering 

Arts (programmes destines aux 
francophones) 

Engineering 

Arts 

Pharmacy 

Commerce 

Library science; Medicine 
Arts (honours only); Nursing 
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Questionnaire on the Characteristics of Effective University Teaching 
Please complete and return to: 

Edward F. Sheffield, 

Professor of Higher Education, 

University of Toronto, 

Toronto 181, Ontario. 

1 Your university (i.e. the one from which you took a bachelor’s or first professional 

degree in 1958, 1963 or 1968): ______ 

2 Degree earned (e.g. B.A., M.D.)_ 3 Year: 19_ 

3 Please name one or more university teachers from whom you took courses leading to 
the degree named in item 2 above whose teaching you considered to be especially ef¬ 
fective. In each case, give his/her name and indicate the course(s) you took from him/her. 

Course Year Approximate 

Name Field (approximation of title) (level) class size 

e.g. R.D. Lawrence History Greek and Roman III 20 

( 1 )- 

( 2 )- 

(3)- 

5 In 200 words, more or less, describe the characteristics, qualities, methods, procedures, 
etc. which, in your view, identify each of these as an excellent teacher. Please describe 
each separately, numbering your descriptions to match the numbers in item 4 above. 
(Use the back of this sheet.) 


6 Your name and address (optional): 
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Appendix B 

Two Notes on the Representativeness of Samples 
(1) The Responding Graduates 

When response in a survey is less than 100% one must ask whether those who responded 
are representative of those who did not. In this case the response rate varied from 9.3% 
of the Laval graduates in administrative science to 29.2% of the honours arts graduates of 
Western Ontario, and was 14.6% overall. There are four kinds of evidence which may be 
relevant. 

The first is that although the response rate for the class of ’58 was 11.4% as compared 
with 15.9% for the class of ’68, the patterns of comments by these two groups are similar 
in most respects. To sharpen this kind of comparison, the patterns of the five faculties 
with the highest response rates (arts at Western Ontario, 29.2%; education at Laval, 
24.7%;agriculture at Manitoba, 21,9%;science at Memorial,21.1%; and nursing at Western 
Ontario, 20.9%) (N = 1,815) were compared with those of the five faculties with the 
lowest response rates (arts at St. Dunstan’s, 9.8%; arts at Ottawa, 9.8%; engineering at 
Nova Scotia Tech, 9.8%; commerce at Sir George Williams, 9.7%; and administrative 
science at Laval, 9.3%) (N = 1,170). Examining only comments which accounted for at 
least three percent of the total number there are two characteristics mentioned more than 
twice as often, proportionately, by one group as the other, and an additional four men¬ 
tioned at least fifty per cent more often by one group than the other: 


Group 


Sub 

-category 

High response rate 

Low response rate 

Ratio 

16 

Experienced in his field 

1.2% 

3.8% 

(3.2 to 1) 

29 

Students expected to do 
much on their own 

2.0% 

4.1% 

(2.0 to 1) 

22 

Concerned for students’ 
progress, etc. 

2.5% 

4.4% 

(1.8 to 1) 

6 

Had a sense of humour, 
amusing 

2.3% 

4.2% 

(1.8 to 1) 

36 

Course organized, 
systematic 

3.2% 

1.8% 

(1.8 to 1) 

17 

Enthusiastic about his 
subject 

4.5% 

1.5% 

(1.5 to 1) 


With respect to these items the respondents may not have been representative of the 
total population. 

The second kind of evidence results from the fact that there were so few responses 
from Memorial science and Saskatchewan pharmacy graduates that they were sent a 
second letter urging them to participate. In both cases the second request elicited new 
responses numbering twice as many as the first. The numbers were still too small for sta¬ 
tistical treatment, but careful reading of the comments made by the two groups revealed 
no noticeable differences in the patterns. Why, then, did the second group not respond to 
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the first request? They did not explain. Lack of interest, maybe? Oversight? Procrastina¬ 
tion? For such mundane reasons, it would seem. 

As the project was being planned, the questionnaire was tried out in a class of 139 
third-year students in arts and science at the University of Toronto. Their response rate 
was 100% because the exercise was undertaken wholly within a class period. This pilot 
project is the third source of evidence. The pattern of their responses was compared with 
that of the graduates in arts and science whose data were used in this study. As has been 
pointed out, there are differences between these two groups but one cannot tell to what 
extent they are related to rate of response. Other explanations seem more satisfactory. 
Besides, the items on which there are observed differences are not the same as those noted 
in the comparison of faculties with higher and lower response rates, above. Even when 
comparisons on this basis are made down to comments constituting no more than one 
per cent of the total (excluding the “other comments” sub-categories) the items identi¬ 
fied in the two series are all different. If the differences in patterns of comments were 
attributable to the influence of the response rate one would expect sameness here rather 
than difference. 

The final type of evidence offered is much less direct. It is, as pointed out above, that 
other approaches to the identification of the characteristics of effective teaching (e.g. the 
SUNY Albany and the University of Toledo studies) have led to much the same list, al¬ 
though the manners of arriving at it have differed significantly. 

Short of surveying the non-respondents themselves, it is unlikely that there can be 
complete assurance that the responses received are representative. What evidence there is, 
though, seems to point in that direction. 

A related question which might well be asked is whether, in the evaluation of teach¬ 
ing, there is a difference between the patterns of responses of students who get high grades 
in their courses and those who get low grades or fail. The data gathered for this study do 
not help to answer that question, but it should be noted that all of the comments re¬ 
ceived were from students who had graduated—whose grades, therefore, had been at least 
satisfactory. Other studies have found no significant difference between good and poor 
students on this point. 

In a Memo to the Faculty from the Center for Research on Learning and Teaching, 
University of Michigan (1965), the evidence is summarized in one sentence: “Rather sur¬ 
prisingly, neither the difficulty of the course nor the grade received is related to the 
pattern of student evaluation.” This is reiterated by Kenneth E. Eble (1970): “The effect 
of grades upon responses to a course or teacher evaluation has been studied as much as 
any single aspect of the subject. Little evidence emerges to support a belief that grades 
determine response.” The question was explored also with CEGEP* students in Quebec 
by Gagne and Chabot (1970). Their finding was the same: 

II s’avere que la satisfaction exprimee par un etudiant a 1’egard de son professeur n’a aucun 
lien avec son succes dans la matiere enseignee par ce professeur, non plus qu’avec sa 
reussite moyenne dans Tensemble des cours. 


* CEGEP: College d’enseignement general et professionnel (offering the 12th and 13th, and for some 
the 14th, years of schooling). 
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(2) The Chosen Professors 

It is interesting to note, too, how typical are the comments which were made about the 
24 chosen professors. Comparison of these (N = 1,062) with the basic profile of all com¬ 
ments about all professors mentioned in all 24 faculties and schools (N = 9,143) reveals 
no noteworthy differences between the two. This encourages the conclusion that, dif¬ 
ferent as the chosen professors are from each other, as a group they are representative of 
the excellent teachers identified in the study. 



